perf: Improve dashboard performance by decreasing rerenders #30958

kgabryje · 2024-11-18T12:50:42Z

SUMMARY

Due to many inefficient redux selectors and other factors causing numerous rerenders, large dashboards were running very slow.

This PR refactors many components related to Dashboard and native filters - some components are rewritten to functional to utilize React's memoization, Redux selectors optimized to select smaller objects that change less frequently, optimized frequently called functions.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

The tests were conducted on a large dashboard - 300+ charts, 40+ filters. Test steps: reload the page, let the 1st tab load, switch to a different tab, let everything load.
Tested on my M2 Pro Macbook - since it's likely that an average Superset user works on a slower computer, I also run the tests with 4x and 20x CPU throttling.
Testing tool - chrome dev tools, performance tab

No throttling
Before - total scripting time 2.73s (screenshot 1), each chart holder blocks the main thread for ~70ms
After - total scripting time 1.48s (screenshot 2), each chart holder blocks the main thread for ~5ms
4x throttling
Before - total scripting time 11.93s (screenshot 3), each chart holder blocks the main thread for ~300ms
After - total scripting time 6.23s (screenshot 4), each chart holder blocks the main thread for ~20ms
20x throttling
Before - total scripting time 82s (screenshot 5), each chart holder blocks the main thread for ~2s
After - total scripting time 32s (screenshot 6), each chart holder blocks the main thread for ~100ms

SCREENSHOTS BEFORE:

No throttling

4x throttling

20x throttling

SCREENSHOTS AFTER:

No throttling

4x throttling

20x throttling

TESTING INSTRUCTIONS

No functional changes - all dashboard functionalities should work like before.

Recommend thorough testing of:

Filter interactions
Chart loading/rendering
Dashboard navigation
Cross-filter functionality
Dashboard state persistence

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

villebro · 2024-11-18T16:08:11Z

These PRs from Kamil always make me soo happy! ❤️

villebro · 2024-11-18T20:28:04Z

/testenv up

villebro

I did my best to find anything to comment or even nit about in this PR, but this looks really solid. I will have to resort to testing to see there's anything that looks out of the ordinary. Will approve once I'm done with the testing rounds..

github-actions · 2024-11-18T20:36:55Z

@villebro Ephemeral environment spinning up at http://52.38.80.189:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

fisjac · 2024-11-18T16:15:01Z

superset-frontend/src/dashboard/components/DashboardBuilder/DashboardBuilder.tsx

-                  </FiltersPanel>
-                );
-              }}
+              {renderChild}


fisjac · 2024-11-18T18:03:32Z

superset-frontend/src/dashboard/components/DashboardBuilder/state.ts

+
+  const toggleDashboardFiltersOpen = useCallback((visible?: boolean) => {
+    setDashboardFiltersOpen(prevState => visible ?? !prevState);
+  }, []);


fisjac · 2024-11-18T19:12:14Z

superset-frontend/src/dashboard/components/SyncDashboardState/index.tsx

+  ],
+  (metadata, dashboardId, colorScheme, filters, dataMask) => {
+    // Building nativeFilters object without spreading in reduce
+    const optimizedNativeFilters = Object.keys(filters).reduce((acc, key) => {


Nice improvement. Nit: should this be named nativeFilters outside of the context of this PR, I don't think "optimized" make sense.

villebro · 2024-11-19T19:42:32Z

/testenv up

github-actions · 2024-11-19T19:54:34Z

@villebro Ephemeral environment spinning up at http://35.91.239.242:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

kgabryje · 2024-11-19T20:25:28Z

@villebro unfortunately the ephemeral environments have been broken for some time now 🙁

kgabryje · 2024-11-19T20:40:48Z

Neat AI summary provided by @rusackas 🙂

Key changes include:

Memoization optimizations:
- Added useMemo/useCallback hooks throughout components
- Memoized selectors with createSelector
- Reduced unnecessary re-renders
Component modernization:
- Converted class components to functional components with hooks
- Removed redundant Redux connections in favor of useSelector/useDispatch
- Simplified prop passing and state management
Architecture improvements:
- Moved Chart container logic directly into component
- Simplified data flow and prop drilling
- Better separation of concerns

The main potential risk areas to watch:

FilterBar and native filtering changes - ensure filter state management remains consistent with previous behavior
Chart component refactoring - verify chart rendering and updates work correctly, especially around:
- Data loading states
- Filter interactions
- Resize handling
Tab/Dashboard navigation - confirm tab state management and routing remains intact

Recommend thorough testing of:
- Filter interactions
- Chart loading/rendering
- Dashboard navigation
- Cross-filter functionality
- Dashboard state persistence

geido

Thank you so much for this! Code looks good but it's a gigantic PR that needs more than a first-pass code review. I'd suggest splitting this into smaller pieces if possible to help with functional tests.

geido · 2024-11-27T17:02:58Z

superset-frontend/src/dashboard/components/gridComponents/Chart.jsx

+  unsetFocusedFilterField,
+} from '../../actions/dashboardState';
+import { changeFilter } from '../../actions/dashboardFilters';
+import { refreshChart } from '../../../components/Chart/chartAction';


Can we use absolute paths?

pull-request-size bot added the size/XXL label Nov 18, 2024

kgabryje marked this pull request as draft November 18, 2024 12:50

dosubot bot added the dashboard:performance Related to Dashboard performance label Nov 18, 2024

kgabryje requested review from geido, justinpark, eschutho, rusackas, villebro and Vitor-Avila and removed request for geido, justinpark and eschutho November 18, 2024 15:27

kgabryje marked this pull request as ready for review November 18, 2024 15:28

dosubot bot added the change:frontend Requires changing the frontend label Nov 18, 2024

kgabryje requested a review from sadpandajoe November 18, 2024 15:28

villebro reviewed Nov 18, 2024

View reviewed changes

fisjac reviewed Nov 18, 2024

View reviewed changes

kgabryje force-pushed the perf/dashboard-speed branch 2 times, most recently from c7aa867 to c8d5eca Compare November 19, 2024 19:16

Optimize SyncDashboardState

edf98c2

kgabryje added 22 commits November 27, 2024 17:27

Rewrite Tabs and DashboardComponent to func to decrease rerenders

949f538

Rewrite Tab to func

130e983

ChartHolder and Dashboard.jsx improvements

59b3f8e

Chart.jsx changes

e9db383

Optimize FilterBar

8162120

Fix some bugs

25764eb

Fix resizing

b15c61b

Fix lint

e338e77

fux lint

8cd379e

Fix tests and lint

023ebe0

restore gitkeep

6bdad2c

bugfix

ac9e923

Add nullchecks

1f51e2e

bugfix

8491de2

name change

51982c0

Fix native filters

4975dcc

Optimize DashboardContainer

46145e4

refactor Row.jsx and getChartIdsInFilterScope

a89e6bc

refactor Column

13b73e4

Optimise cross filters selector

c4190d6

Add props equal func to Chart memo

d7019d6

Fix unnecessary Chart rerender when tab is changed

26a1163

kgabryje force-pushed the perf/dashboard-speed branch from 555df32 to 26a1163 Compare November 27, 2024 16:29

lint fix

acd0935

geido approved these changes Nov 27, 2024

View reviewed changes

sadpandajoe requested review from villebro and fisjac November 27, 2024 19:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: Improve dashboard performance by decreasing rerenders #30958

perf: Improve dashboard performance by decreasing rerenders #30958

kgabryje commented Nov 18, 2024 •

edited

Loading

villebro commented Nov 18, 2024

villebro commented Nov 18, 2024

villebro left a comment

github-actions bot commented Nov 18, 2024

fisjac Nov 18, 2024

fisjac Nov 18, 2024

fisjac Nov 18, 2024

villebro commented Nov 19, 2024

github-actions bot commented Nov 19, 2024

kgabryje commented Nov 19, 2024

kgabryje commented Nov 19, 2024

geido left a comment

geido Nov 27, 2024

perf: Improve dashboard performance by decreasing rerenders #30958

Are you sure you want to change the base?

perf: Improve dashboard performance by decreasing rerenders #30958

Conversation

kgabryje commented Nov 18, 2024 • edited Loading

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

villebro commented Nov 18, 2024

villebro commented Nov 18, 2024

villebro left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 18, 2024

fisjac Nov 18, 2024

Choose a reason for hiding this comment

fisjac Nov 18, 2024

Choose a reason for hiding this comment

fisjac Nov 18, 2024

Choose a reason for hiding this comment

villebro commented Nov 19, 2024

github-actions bot commented Nov 19, 2024

kgabryje commented Nov 19, 2024

kgabryje commented Nov 19, 2024

geido left a comment

Choose a reason for hiding this comment

geido Nov 27, 2024

Choose a reason for hiding this comment

kgabryje commented Nov 18, 2024 •

edited

Loading